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THE  MISSION  OF  AGARD 


The  mission  of  AGARD  is  to  bring  together  the  leading  personalities  of  the  NATO  nations  in  the  Helds  of  science 
and  technology  relating  to  aerospace  for  the  following  purposes: 

- Exchanging  of  scientific  and  technical  information; 

- Continuously  stimulating  advances  in  the  aerospace  sciences  relevant  to  strengthening  the  common  defence 
posture; 

- Improving  the  co-operation  among  member  nations  in  aerospace  research  and  development; 

- Providing  scientific  and  technical  advice  and  assistance  to  the  North  Atlantic  Military  Committee  in  the  field 
of  aerospace  research  and  development; 

- Rendering  scientific  and  technical  assistance,  as  requested,  to  other  NATO  bodies  and  to  member  nations  in 
connection  with  research  and  development  problems  in  the  aerospace  field; 

- Providing  assistance  to  member  nations  for  the  purpose  of  increasing  their  scientific  and  technical  potential; 

- Recommending  effective  ways  for  the  member  nations  to  use  their  research  and  development  capabilities  for 
the  common  benefit  of  the  NATO  community. 

The  highest  authority  within  AGARD  is  the  National  Delegates  Board  consisting  of  officially  appointed  senior 
representatives  from  each  member  nation.  The  mission  of  AGARD  is  carried  out  through  the  Panels  which  are 
composed  of  experts  appointed  by  the  National  Delegates,  the  Consultant  and  Exchange  Programme  and  the  Aerospace 
Applications  Studies  Programme.  The  results  of  AGARD  work  are  reported  to  the  member  nations  and  the  NATO 
Authorities  through  the  AGARD  series  of  publications  of  which  this  is  one. 

Participation  in  AGARD  activities  is  by  invitation  only  and  is  normally  limited  to  citizens  of  the  NATO  nations. 


The  content  of  this  publication  has  been  reproduced 
directly  from  material  supplied  by  AGARD  or  the  author. 
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PREFACE 


The  use  of  computerised  techniques  of  structural  analysis  is  now  standard  in  many  branches  of  engineering.  There 
is,  however,  a wide  range  of  programs  available  both  commercially  and  within  individual  organisations.  These  programs 
differ  in  their  capabilities  and  in  their  costs  and  ease  of  use.  The  potential  user  may  experience  considerable  difficulty 
in  selecting  a program  that  is  appropriate  to  his  particular  class  of  work. 

An  Ad  Hoc  Group  to  consider  this  problem  was  established  by  the  Structures  and  Materials  Panel  at  its  45th 
Meeting.  Following  discussions  at  the  46th  Meeting  in  Aalborg,  Denmark,  it  was  decided  to  invite  the  presentation  of 
specialist  Pilot  Papers  to  give  guidance  to  both  the  Panel  and  to  users.  The  papers,  that  were  subsequently  presented 
at  the  47th  Meeting  in  Florence,  Italy,  by  Mr  Andrew  and  Mr  Taig  were  judged  to  be  of  such  general  interest  that 
they  warrant  wide  distribution. 

The  paper  by  Mr.  Andrew  describes  in  detail  the  technical  and  administrative  course  of  action  that  has  been 
adopted  by  a major  industrial  organisation  to  select  and  implement  programs  that  are  appropriate  to  its  work.  Mr  Taig 
presents  a similar  discussion  but  with  perhaps  more  emphasis  on  technical  issues. 

Since  many  of  the  essential  elements  of  the  selection  process  appear  to  be  largely  subjective  judgements,  it  is  felt 
that  little  further  action  can  be  taken  by  the  Panel.  The  papers  are  therefore  published  to  give  assistance,  and  perhaps 
solace,  to  those  charged  with  the  process. 


J.A.DUNSBY, 

Chairman, 

Ad  Hoc  Group  on  Structural  Analysis  Computer  Programs. 
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SUMMARY 


The  proliferation  of  large  scale  structural  analysis  programs  prior  to  1973  led  to  the 
formation  of  Rockwell's  Ad-Hoc  Subcommittee  for  Computerized  Structural  Analysis  (SCSA), 


This  paper  describes  how  the  Ad-Hoc  SCSA  scoped  the  task  of  evaluating  the  computer 
programs,  how  it  developed  the  basis  for  its  evaluations  and  recommendations  and 
presents  tables  that  define  the  grading  system  that  emerged.  It  also  describes  the 
compilation  of  the  final  report  which  still  serves  as  a guide  for  the  permanent  SCSA 
formed  in  1974.  The  SCSA  formed  ASKA  and  NASTRAN  Configuration  Control  Boards  to 
control  the  maintenance  and  development  of  the  programs.  The  function  of  these  boards 
in  certification  of  the  computer  programs  and  the  function  of  the  SCSA  in  Rockwell 
Group  Services  Project  reviews  are  d.scussed.  Finally,  some  recommendations  are  made 
to  those  who  must  select  computer  programs  from  those  that  are  available. 


INTRODUCTION 

In  the  years  following  1966,  when  Computer  Sciences  Corporation,  with  MacNeal  Schwendler 
and  Martin  Baltimore  as  subcontractors,  started  the  development  of  NASTRAN,  there  was  a 
proliferation  of  large  scale  structural  analysis  programs.  Boeing's  ATLAS  program, 

Me  Donnell  Douglas’  FORMAT  program,  Mechanics  Research's  STARDYNE  program  and  the 
Institute  for  Static  and  Dynamic  Analysis  of  Aerospace  Vehicles’  ASKA  program  are 
examples  (see  Reference  1 for  a survey  of  some  500  such  programs).  Of  the  two  methods 
that  were  applicable  to  large  scale  finite  element  models  of  structures,  namely  the 
force  method  and  the  displacement  method,  the  latter  emerged  as  the  successful  one  and 
is  now  used  in  most  general  purpose  programs.  On  the  other  hand  the  force  method  has 
such  merits  as  greater  efficiency  in  special  applications  that  make  it  the  preferred 
one  for  these  special  cases. 

NASTRAN  eventually  came  to  use  only  the  displacement  method  and  was  released  to  the 
public  in  November  1970.  However,  it  had  many  limitations  at  that  time  so  Rockwell 
acquired  ASKA  for  use  on  its  major  systems  and  first  applied  it  to  the  wing  structure 
of  the  B-l.  Even  though  various  improvements  of  ASKA  were  still  being  received  in 
July  1972  when  the  authorization  to  proceed  with  the  Shuttle  was  received  at  the  Space 
Division,  Rockwell  decided  to  use  it  for  the  entire  development  because  of  ASKA's  multi- 
level substructuring  capability.  Several  other  methods  had  been  mechanized  to 
analyze  thin  shells,  thick  shells,  piping  systems,  composite  structures,  etc.  Still 
other  programs  had  been  generated  to  pre  and  post-process  data  that  were  used  by  the 
main  processors;  some  of  these  were  for  such  special  purposes  as  fatigue  and  fracture 
mechanics,  aero-elastic  analyses,  etc. 

At  about  this  time  plans  were  being  made  to  integrate  large  scale  steady  and  unsteady 
aerodynamics  programs  and  thermodynamics  programs  with  the  structural  analysis  programs, 
using  a master  program  and  interactive  graphics  to  interface  between  them  to  speed  up 
the  design  process.  Because  of  this  continuing  growth  of  scope  of  computer  programs, 
it  was  considv  -ed  advantageous  to  form  the  Subcommittee  for  Computerized  Structural 
Analysis  (SCSA). 

The  Ad-Hoc  Subconenittee  for  Computerized  Structural  Analysis 

In  February  1973  Rockwell  International's  Structures  Panel  formed  the  Ad-Hoc  SCSA.  The 
charter  contained  the  following  paragraph: 

The  objective  of  the  subcommittee  is  to  review  structural  analysis  compucer 
programs  (static  and  dynamic)  existing  at  the  divisions  in  the  North  American 
Aerospace  Croup  and  the  Electronics  Group  and  to  plan  and  recommend  such  amend- 
ments, additions,  and  improvements  as  it  shall  consider  essential  for  adequate 
accuracy,  efficiency,  and  turnaround  time  in  applications  during  the  design 
cycle  of  aerospace  systems.  Included  in  the  review  shall  be  ASKA  and  NASTRAN, 
the  MMLS  concept  (at  the  Space  Division),  the  subject  of  Interactive  Graphics, 
as  well  as  other  pertinent  static  and  dynamics  programs  in  the  Divisions.  The 
subcommittee  will  also  serve  as  the  North  American  Aerospace  Group/Electronic 
Group's  focal  point  for  ASKA  and  NASTRAN  maintenance  and  improvement  activity, 
information  exchange  on  all  structural  software  programs,  and  will  generate 
technical  advice  and  counsel  for  the  use  of  Rockwell  International's  representa- 
tive on  the  NASA/Industry  NASTRAN  Advisory  Board. 


The  subcommittee  consisted  of  eight  people,  two  representing  the  B-l  Division,  two 
representing  the  Space  Division,  and  one  representing  each  of  the  Tulsa,  Rocketdyne, 
Columbus  and  Autonetics  Divisions.  The  author  was  appointed  the  chairman  and  was  given 
approximately  five  months  to  accomplish  the  assigned  task. 

I.  Scope  of  the  Evaluations 

The  first  of  eight  meetings  was  held  to  acquaint  all  the  Subcommittee  members  with  the 
large  scale  computer  programs  that  were  then  under  development  or  in  use  by  government 
agencies  and  by  Rockwell  and  other  aerospace  companies.  Special  emphasis  was  placed  on 
the  problems  users  found  when  using  these  programs.  This  meeting  ended  with  open 
discussions  about  defining  the  scope  of  our  evaluations. 

The  second  meeting  was  held  to  acquaint  the  members  with  the  design  analyses  as  conduct- 
ed at  Rockwell  and  to  acquaint  them  with  upcoming  computer  hardware  systems.  One  type 
of  integration  program,  intended  for  preliminary  design  analyses,  was  under  development 
at  the  B-l  Division,  namely,  the  Rapid  Response  Analysis  Program  for  Integrated  Design. 
Another  type  of  integration  program,  intended  for  intermediate  and  final  design 
analyses,  was  under  development  at  the  Space  Division,  namely,  the  Model-Modal-Loads- 
Stress  program.  The  subcommittee  heard  descriptions  of  these  programs  and  of  the 
Integrated  Programs  for  Aerospace-Vehicle  Design  program.  The  IPAD  program  was  in  the 
conceptual  stage  and  was  under  development  for  the  NASA. 

At  the  third  meeting  each  member  presented  his  evaluation  of  the  completeness  of  the 
research  phase  tentatively  just  completed  and  his  recommendations  for  completing  the 
work  of  the  subcommittee.  Eight  questions  were  suggested  as  a possible  format  for  those 
presentations : 

1.  Are  RRAPID  and  MMLS  competitors  to  various  levels  of  IPAD?  Is  it  likely 
that  eventually  they  will  be  replaced  by  IPAD?  How  much  effort  should 
go  into  synthesizing  these  systems  of  programs? 

2.  Which  unsteady  and  quasi-steady  aerodynamics  methods  should  be  relied 
upon  to  do  the  loads  and  flutter  analysis  jobs? 

3.  In  view  of  the  uncertainty  about  the  impact  that  future  computers  (those 
with  virtual  memories)  will  have  on  the  operation  of  the  ASKA  program, 
what  level  of  priority  should  be  assigned  for  its  development  and 
maintenance  relative  to  that  for  NASTRAN?  Do  we  need  both  ASKA  and 
NASTRAN? 

4.  What  should  be  dene  about  the  lack  of  optional  procedures  for  generating 
reduced  order  SIC's  and  consistent  mass  matrices? 

5.  What  is  the  future  of  computer  programs  that  use:  a)  the  force  method,  or 
b)  a mixed  force  and  displacement  method?  Could  some  of  the  techniques 
that  make  certain  of  these  programs  extremely  efficient  be  applied  equally 
well  to  programs  that  use  the  displacement  method? 

6.  What  can  be  done  to  eliminate  the  difficulties  of  modifying  NASTRAN? 

7.  Should  we  recommend  establishing  a corporate  subcommittee  of  experts  in 
the  use  of  NASTRAN,  ASKA,  etc.? 

8.  Who  should  be  assigned  responsibility  for  maintaining  the  programs  on  the 
structures  disc  pack? 

These  questions  elicited  some  lively  discussions  which  led  to  two  more  questions  the 
subcommittee  decided  it  should  answer  in  the  final  report: 

9.  What  do  wo  recommend  regarding  the  use  of  interactive  graphics? 

10.  Wh..*t  is  our  recommendation  regarding  user's  manuals? 

We  also  decided  that  we  had  a large  enough  set  of  computer  programs  to  make  meaningful 
evaluations  and  recommendations. 

II.  Basis  of  the  Recommendations 

At  the  fourth  meeting  the  members  listed  the  capabilities  and  limitations  of  the 
previously  reviewed  computer  programs  and  each  member  listed  the  key  future  needs  of  his 
division.  They _ separated  these  needs  into  one  year,  three  year,  and  five  year  needs. 

(A  manifest  basis  for  making  recommendations  was  beginning  to  emerge.)  Members  then 
presented  evaluations  ot  tbe  computer  programs  relative  to  the  key  future  needs  of  each 
of  the  divisions.  They  used  a form  that  had  been  suggested  by  one  of  the  members  and 
presented  these  at  the  fifth  meeting.  Also,  the  interim  report  was  read  and  unanimously 
approved  et  this  meeting. 


3 


An  excerpt  from  the  interim  report  reads: 

The  Ad-Hoc  Subcommittee  has  already  drawn  firm  conclusions  on  certain  points.  It 
believes  there  will  be  a continuing  need  for  coordinating  the  in-house  development 
;»n<i  maintenance  of  large  scale  computer  programs,  specifically,  both  NASTRAH  and 
ASKA.  Coordination  is  necessary  to  assure  maximum  utility  for  all  the  groups  of 
users.  Also,  the  Subcommittee  is  keenly  aware  that  within  its  short  life  span 
it  can  dono  more  than  base  its  recommendations  on  the  current  state  of  a technol- 
ogy that  is  undergoing  tremendous  changes  and  growth.  Continuing  surveillance  of 
the  technology  will  be  necessary  to  a viable  program.  Therefore,  it  recommends: 

1.  That  the  Rockwell  International  Structures  Technical  Pane},  establish  a 
permanent  Subcommittee  for  Computerized  Structural  Analysis. 

2.  That  the  permanent  subcommittee  be  assigned  the  responsibility  to  identify 
engineer /programmer  experts  in  the  use  and  modification  of  NASTRAN,  ASKA,  and 
other  selected  computerized  structural  analysis  programs. 

3.  That  the  function  of  a configuration  control  board  of  computer  programs  in 
multi-divisional  use  be  assigned  to  the  subcommittee. 

A.  That  improvements  be  male  of  pre-  and  post-processors  for  HAS TRAN  and  ASKA. 

III.  The  Final  Report  (Reference  2) 

The  group  spent  a considerable  part  of  the  sixth  meeting  in  a workshop  type  of  prepara- 
tion of  composite  evaluation  charts  of  the  twelve  selected  computer  programs.  These 
were  selected,  of  course,  from  programs  available  at  nominal  cost  to  Rockwell.  Two 
systems  of  computer  programs,  MMLS  and  RRAPID  were  also  evaluated  but  not  included  on 
the  evaluation  charts.  These  charts  are  presented  here  as  Tables  I through  IV  which, 
with  some  study,  should  be  self  explanatory.  It  should  be  remembered  that  these  evalua- 
tions were  made  in  early  1973,  and  there  ha^e  been  some  significant  improvements  made 
to  some  of  the  programs  since  then. 

The  charts  list  evaluations  of  specific  items  in  the  following  categories: 

1.  Computer  hardware  requirements  and  compatibility  with  future  hardware. 

2.  Levels  of  analysis  requirements  that  are  satisfied  by  the  program. 

3.  Probability  that  the  program  will  be  maintained  and  further  developed. 

A.  Adequacy  of  pre-  and  post-processors. 

5.  Adequacy  of  documentation. 

6.  Adequacy  of  library  of  structural  elements. 

7.  Efficiency  of  the  program. 

8.  Capability  to  perform  structural  dynamics  analyses. 

The  narrative  descriptions  and  evaluations  of  the  programs  were  all  done  in  the 
following  format: 

a.  Program  Description  and  Capabilities 

b.  History  of  Applications 

c.  Personnel  Availability  to  Maintain  and  Develop  the  Program 

d.  Users;  Identified  by  Division 

e.  Limitations 

f.  Documentation 

The  conclusions  and  recommendations  were  compiled  at  the  seventh  meeting.  Also,  we 
decided  that  the  final  report  should  contain  the  answers  to  the  ten  questions  we  had 
posed,  and  that  the  evaluations  of  computer  programs  and  recommendations  for  additional 
work  should  be  presented  in  both  narrative  and  tabular  form;  the  latter  table  should 
list  projects  in  prioritized  order.  Two  more  workshop  type  meetings  were  held  to 
compile  and  proofread  the  final  draft. 

The  recommendations  in  the  interim  report  were  reiterated  in  the  final  report  along  with 
the  other  recommendations.  The  final  report  was  well  received  by  th'-*  Structures  Panel 
and  by  Corporate  executives  and  it  was  used  to  define  the  Group  S^ivices  Projects  that 
are  nerformed  as  corporate  efforts.  After  some  delay,  caused  partly  by  reorganization 
of  the  corporate  structure,  the  permanent  SCSA  was  formtd. 
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The  Permanent  SCSA 


In  December  1974  the  Rockwell  Structures  Technical  Panel  authorized  the  permanent  SCSA. 
The  charter  starts  as  follows: 

Because  of  the  need  for  more  interdivisional  cooperation  among _ structural  analysts 
to  share  information,  expertise,  computer  programs,  program  maintenance  and  program 
development  it  is  considered  advantageous  to  establish  a Subcommittee  for 
Computerized  Structural  Analysis.  The  subcommittee  will  identify  the  engineer/ 
programmer  experts  it  believes  are  qualified  to  satisfy  these  needs  and  the 
subcommittee  will  act  as  a permanent  board  to  control  the  improvements  and  develop- 
ment of  computer  programs  in  multi-divisional  use. 

Objective 

Implement  the  general  recommendations  contained  in  the  final  report  of  the  Ad  Hoc 
Subcommittee  for  Computerized  Structural  Analysis  (report  no.  NA73-556) , as 
modified  by  action  of  the  Structures  Technical  Panel  (Addendum  I to  NA73-556). 

Monitor  the  recommended  development  effort  for  those  programs  which  the  Structures 
Technical  Panel  has  selected  for  Group  Service  action. 

Maintain  surveillance  of  changes  to  the  key  needs  of  the  divisions  relative  to 
impact  on  the  objectives  of  the  Subcommittee. 

As  expected,  there  were  many  unsolved  problems  that  had  accumulated  since  the  dissolu- 
tion of  the  Ad-Hoc  Subcommittee  so  the  first  year  was  a very  busy  one  for  the  SCSA. 

One  of  the  problems  was  a continuing  lack  of  adequate  documentation  and  other  types  of 
communication  by  the  program  developers  and  the  user  community.  Another,  was  that  the 
members  felt  that  the  number  of  members  was  not  weighted  properly,  either  relative  to 
the  number  of  the  users  at  the  divisions  or  relative  to  the  technical  disciplines  of 
the  users.  As  a result  the  membership  was  increased  by  two  representing  the  B-l 
Division  and  two  representing  the  Space  Division.  This  group  then  re-examined  its 
recommendations  of  nearly  two  years  past,  made  a few  changes  in  emphasis  but  essentially 
reiterated  the  recommendations  of  the  Ad-Hoc  Subcommittee.  The  group  also  restated  its 
recommendation  that  Configuration  Control  Boards  be  established  for  both  NASTRAN  and 
ASKA  and  that  each  of  the  participating  divisions  be  represented  on  the  boards. 

I.  ASKA  and  NASTRAN  Configuration  Control  Boards  (CCB's) 

Rockwell  has  a documented  procedure  which  authorizes  these  boards  and  also  establishes 
their  function  and  reporting  level.  Following  the  authorization  the  SCSA  drew  up  two 
documents  for  each  board:  1)  Research  and  Engineering  Standards  - A5KA/NASTRAN 
Program  Configuration  Management  Plan,  and  2)  Research  and  Engineering  Procedures  - 
ASKA/NASTRAN  Program  Configuration  Management.  These  documents  were  approved  and 
implemented  in  mid  1976. 

The  subcommittee  decided  early  in  the  life  of  the  CCB's  that  each  division  member  must 
have  the  freedom  to  copy  the  ’’standard"  versions  and  modify  the  copies  for  special 
purposes.  Otherwise,  the  "standard"  version  is  used  throughout  the  divisions  that  have 
access  to  the  computing  center. 

The  NASTRAN  CCB  recently  received  level  16.0  of  NASTRAN  from  C0SMIC.  This  release 
represents  the  turning  point  in  the  comparative  utility  of  NASTRAN  because  of  its  new 
capability  to  perform  multi-level  substructuring.  It  also  points  up  the  continuing 
need  for  the  CCB  and  the  subcommittee,  as  computer  programs  continue  to  be  developed. 

The  operation  of  the  NASTRAN  CCB  is  described  in  Reference  3,  from  which  the  following 
paragraph  was  taken: 

The  representatives  of  the  participating  divisions  are  the  members  that  evaluate 
the  proposed  changes  for  impact  to  an  on-going  project.  They  make  recommenda- 
tions for  improvements  and  submit  developments  for  incorporation  into  the 
Rockwell  NASTRAN  program.  They  submit  recommendations  for  inclusion  in  the 
NASTRAN  Group  Services  statement  of  work  proposal.  Each  representative  serves  as 
a focal  point  at  his  respective  division  for  dissemination  of  NASTRAN  documenta- 
tion, the  initial  evaluation  of  NASTRAN  user's  problems,  and  the  coordination  of 
change  requests  submitted  to  the  CCB. 

One  of  the  CCB's  function  is  to  certify  any  modifications  made  to  the  Rockwell  standard 
version  of  the  program.  Much  of  the  material  in  the  next  two  paragraphs  was  taken 
from  Reference  4. 

Complete  certification  of  large  scale  computer  programs  is  the  elusive  goal  of  all  soft- 
ware engineers.  However,  ia  a. recent  conference  on  computerized  structural  analysis, 
the  U.  S.  Bureau  of  Standards  indicated  that  it  took  upward  of  10  manhours  of  effort  to 
certify  a "small"  computer  program.  To  undertake  such  a certification  effort  the 
version  of  the  program  is  usually  frozen  while  the  prescribed  checks  are  being  made. 

A more  practical  approach  in  industrial  usage  is  to  accept  a probabilistic  certifica- 
tion, namely  debugging  pilot  versions.  This  approach  was  taken  to  evolve  present  day 
FORTRAN  compilers  and  to  evolve  the  NASTRAN  program  itself. 


The  procedure  followed  at  Rockwell  to  “certify"  the  NASTRAN  and  ASKA  programs  is  to 
verify  that  the  set  of  demonstration  problems  supplied  by  COSMIC,  as  well  as  a set  of 
benchmark  problems  developed  at  Rockwell,  yield  correct  solutions.  A constant  effort  is 
made  to  include  in  this  set  of  benchmark  problems  a spectrum  of  production  pioblems  that 
have  been  encountered  and  solved  at  Rockwell,  plus  those  that  test  the  various  capabil- 
ities of  the  programs.  The  goal  is  to  ensure  the  highest  probability  of  success 
before  the  modified  versions  are  released  for  production  use. 

II.  Group  Services  Project  Reviews  and  Recommendations 

The  SCSA  assumed  the  job  of  performing  the  overall  technical  review  of  Group  Serv  es 
Projects  related  to  structural  analysis.  It  also  concluded  that  to  perform  the  rt^iew 
adequately,  a much  more  detailed  and  timely  job  of  reporting  by  the  Project  Managers 
was  necessary.  Part  of  the  requirement  is  that  the  SCSA  must  make  its  recommendations 
six  months  in  advance  of  funding  allocations,  and  must  base  those  recommendations 
partly  on  past  performance  of  the  Project  Manager. 

When  formulating  this  year's  recommendations  for  FY  ‘79  Projects,  the  SCSA  observed  that 
even  though  it  is  apparent  that  much  needs  to  be  done  in  the  field  of  interactive 
graphics,  it  is  proceeding  very  slowly  because  of  lack  of  necessary  hardware. 

III.  Future  Activities  of  SCSA 

Another  committee  at  Rockwell  that  has  responsibility  for  acquisition  of  computing 
hardware  (the  Engineering  Computing  Policy  Board),  has  addressed  itself  to  the  shortage 
of  interactive  graphic  hardware  and  the  SCSA  will  be  coordinating  its  activities  with 
their's  during  the  hardware  acquisition. 

The  SCSA  will  continue  its  annual  review  of  Group  Services  Projects  and  make  recomnenda- 
tions  for  maintenance  and  development  of  NASTRAN,  and  probably  just  maintenance  of 
ASKA.  It  will  also  conduct  at  least  biennial  reviews  of  the  key  future  needs  of  the 
represented  divisions  of  Rockwell;  more  often,  when  major  projects  are  cancelled  or 
acquired. 

Recommendations 


At  the  risk  of  being  pedantic,  the  following  recommendations  are  made  to  those  who  have 

the  job  of  selecting  computer  programs. 

1.  In  the  interest  of  making  unanimously  endorsed  and  meaningful  recomnendations,  keep 
the  Ad-Hoc  Group  small.  Rockwell's  Ad-Hoc  group  had  nine  members  and  we  had  to 
make  a deliberate  effort  to  rework  each  recommendation  until  all  members  could 
endorse  it.  A much  larger  group  might  never  have  done  it. 

2.  Keep  the  period  of  performance  of  the  Ad-Hoc  group  short  in  the  interest  of  making 
a concentrated  effort  to  accomplish  its  task. 

3.  Define  the  immediate,  intermediate  and  long  range  needs  of  the  community  the 
Ad-Hoc  group  represents  and  start  small  when  defining  that  community. 

4.  By  means  of  a preliminary  evaluation,  select  the  smallest  group  of  computer 
programs  that  will  provide  flexibility  of  choice  after  a detailed  evaluation. 

Both  evaluations  should  be  made  in  terms  of  established  needs. 

The  grading  system  shown  in  the  Tables  is  suggested  as  one  that  is  general  enough 
so  that  a group  can  arrive  at  a consensus. 

5.  Document  everything  that  led  to  the  recomnendations.  Years  later  it  will  help 
put  the  recomnendations  in  perspective. 

These  are  recommendations  to  those  who  must  select  from  the  libraries  of  programs. 

Those  who  accumulate  the  libraries  have  different  objectives,  and  they  may  approach 

them  in  much  different  ways  than  outlined  above. 
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** Status  information  on  pre-processors,  poet-processors,  and  main  processors  are  identical 
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SELECTION  CRITERIA  FOR  STRUCTURAL  ANALYSIS  PROGRAMS 
by  I.C.  Taig 

Coordinator  Advancad  Airframe  Technology 
Britiah  Aerospace,  Aircraft  Group, 

Wart  on  Division, 

Warton,  Preston,  Lancs  PR4  1AX 
England 


Introduction 

This  paper  is  presented  on  the  premise  that  a prospective  user  has  several  candidate 
structural  analysis  programs  in  mind  and  that  their  relative  merits  and  demerits  are  not 
so  obvious  as  to  make  selection  a fairly  trivial  matter.  It  is  further  supposed  that 
it  is  required  to  embark  on  a fairly  formal  procedure  to  select  the  "best"  system  in  a 
manner  illustrated  conceptually  in  Fig.l.  Given  e number  of  candidate  systems  and 
informed  management,  selection  could  be  based  on  the  first  and  last  steps  in  this 
procedure,  l.e.  the  initial  rejection  of  totally  unsuitable  aystsma  and  choice  from 
among  the  remainder  on  the  basis  of  personal  judgment  alone.  This  is  a feasible  and 
often-used  approach  but  for  the  purpose  of  this  discussion  a three-stage  procedure  is 
assumed 

1.  Initial  screening 

2.  Formal  assessment 

3.  Management  decision 

The  paper  is  aimed  primarily  towards  users  in  large  industrial  organisations  and 
makes  particular  reference  to  the  use  of  structural  analysis  programs  in  inter-company 
and  international  joint  projects. 
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1 . INITIAL  SCREENING 


There  are,  at  present,  many  structural  analysis  programmes  available,  either  for 
commercial  sale  or  rental,  or  for  distribution  through  public  bodies.  Of  these  there 
are  at  least  15  systems  fairly  widaly  available  in  NATO  countrlss  which  are  credible 
contenders  for  use  by  major  companies.  Furthermore,  srast  large  companies  today  have  an 
ln-houae  system  of  their  own  or  have  access  to  such  a system  in  a partner  organisation. 

The  problem  of  selection  is  therefore  a very  real  one  but  it  is  not,  of  course,  necessary 
to  cut  the  choice  down  to  a single  system.  Often  a company  will  opt  to  use  and  suilntain 
two  or  more  systems  whose  strengths  lie  in  different  areas,  so  that  together  they  provide 
adequate  capability. 

Before  discussing  the  initial  screening  for  acceptable  candidate  systems  it  is  worth 
making  an  important  philosophical  point.  Most  of  the  currently  available  major  systems 
ars  good  and  ara  developed  by  competent  and  enthusiastic  teams.  A large  amount  of  time 
and  effort  could  be  spent  in  drawing  up  a detailed  comparison  between  such  systems  only 
to  obtain  an  inconclusive  result  and  intuitive  judgsmnt  would  still  be  used  in  the  final 
decision.  The  approach  suggested  here  is  to  elicit  pertinent  facts  with  as  little 
effort  as  possible,  to  reduce  contenders  wherever  possible  by  avoiding  major  obstacles 
to  successful  implementation  and  to  make  a final  decision  on  the  basis  of  judgment 
supported  by  an  objective  assessment  of  the  facta.  TIME  WILL  BE  BETTER  SPENT  IN  CARE- 
FUL IMPLEMENTATION  OF  ANY  GOOD  SYSTEM  THAN  IN  CONDUCTING  AN  OVER-ELABORATE  COMPARISON 
BETWEEN  NEARLY  EQUAL  CONTENDERS. 

The  first  step  in  the  process  is  to  eliminate  those  systems  which  do  not  satisfy 
important  user  requirements  and  cannot  readily  be  adapted  to  do  so.  The  criteria  used 
st  this  stage  Include  the  followlng:- 

to  potential  users  on  a sound  commercial  basis 
in  the  future  consistent  with  National  policies 

with  the  eosq>any's  computer  configuration  (and  those 
of  partners) 

with  the  operating  system  and  software  used  by  the  company 
with  the  swde  of  operation  deetanded  by  the  user 

to  handle  problems  of  the  types  required 
to  cope  with  the  sice,  complexity  and  throughput  needed 
consistency  with  national  or  company  policy; 

cost  limitations  (unlikely  to  rule  out  a system  at  thie  stage) 

It  is  advisable  to  seek  assurance  of  the  continued  availability  of  rented  or 
leased  systems  and  bureau  services  and  of  technical  support , irrespective  of  method  of 
acquisition,  for  as  long  as  the  user  thinks  necessary.  This  is  particularly  true  of 


Availability* 


Compatibility ' 


Capability 
Capacity  — 


Comamrclal  constraints ' 
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programs  supplied  through  public  institutions  in  other  countries,  which  can  be  subject 
to  changing  political  directives. 

Compatibility  with  operating  systesis  and  software  can  obviously  be  achieved  at  a 
cost  - at  this  stage  it  would  be  appropriate  to  retain  an  otherwise  satisfactory 
candidate  system  and  simply  record  t.te  software  modification  cost  for  use  in  subsequent 
assessments.  Incompatibility  with  the  computer  hardware  should  normally  be  regarded 
as  reason  for  rejecting  a candidate  system  outright.  The  cost  of  rewriting 
incompatible  code  is  too  high  to  justify  further  attention  unless  there  is  no  other 
possible  candidate  system. 

Again,  if  the  user  organisation  is  set  up  in  such  a manner  that  interactive 
operation  from  terminalj  is  essential,  there  is  no  point  in  giving  detailed  attention 
to  systems  which  cannot  conceivably  operate  in  this  mode. 

Capacity  may  be  very  hard  to  Judge  by  discussion  with  the  potential  supplier  alone. 
It  is  best,  if  in  doubt,  to  try  to  find  other  users  and  obtain  first  hand  experience, 
failing  this  ask  the  supplier  to  demonstrate  the  ability  to  handle  marginal  tasks. 

The  starting  list  of  candidates  may  by  now  have  been  reduced  to  a manageable  number 
and  the  next  section  deals  with  their  more  detailed  appraisal. 

2.  FORMAL  ASSESSMENT 


The  three  principal  elements  in  the  formal  assessment  of  competing  programs  are 
described  in  Fig.l  as 

- Selection  criteria 

- Ranking  method 

- User  experience 

The  main  purpose  of  this  paper  is  to  deal  with  the  criteria  but  before  doing  so  a 
few  words  on  the  other  items  are  appropriate.  It  is  already  evident,  and  will  quickly 
becosM  more  so,  that  any  comparison  will  involve  differing  levels  of  performance  against 
criteria  which  have,  superficially,  nothing  in  common.  The  basic  dilesima  in  all  formal 
assessments  is  to  compare  the  dissimilar  in  some  meaningful,  overall  way.  If  the  user's 
task  is  sufficiently  explicit,  many  of  the  conflicting  requirements  can  be  reduced  to 
the  two  main  programme  control  factors 

- what  is  the  total  cost  of  doing  all  the  jobs  needed? 

what  will  be  the  elapsed  t ime  (or  the  rate  of  turnover)? 

In  this  situation  it  is  only  necessary  to  make  one  major  subjective  Judgment  (the 
monetary  value  of  tiaw)  to  reduce  all  sssessawnts  to  a common  currency. 

More  often  no  such  facile  solution  presents  Itself  and  the  only  way  to  introduce 
formality  into  the  assessmsnt  is  to  make  several  Independent  assessments  of  "value"  to 
the  organisation  of  good  performance  against  specific  critarla.  These  can  be  assessed 
by  a "points  scheme"  or  by  any  grading  symbols,  whereupon  a further,  subjective, 
weighting  judgment  must  be  applied  to  obtain  an  overall  ranking.  The  result  is  multiply 
suspect  and  only  the  very  unwise  would  use  such  a ranking  as  the  only  basis  for  choice. 
However,  this  approach,  uesd  by  people  of  good  sense,  often  produces  results  which  are 
not  unduly  sensitive  to  substantial  changes  of  individual  weightings.  A competent 
manager  will  ask  for  an  assessment  of  sensitivity  to  changes  in  the  more  significant 
parameters  and  will  then  have  a useful  background  for  the  exercise  of  his  judgment. 

These  points  will  be  illustrated  later  by  a simple  example. 

The  experience  of  other  users  in  a similar  type  of  business  is  invaluable  in 
supporting  or  modifying  the  claims  of  suppliers  regarding  the  usefulness  of  a program 
in  practice.  In  particular,  only  a user  can  comment  adequately  on  the  ease  of  use  of 
a system,  the  intelligibility  of  its  documentation  and  the  amount  of  internal  support 
which  it  demands  in  order  to  function  satisfactorily. 

The  selection  criteria  are  subdivided  for  convenience  into  three  groups,  depending 
upon  the  principal  objectives  which  they  aim  to  satisfy.  These  are  described  in 
fig.  I ae:- 


2.1  Technical  Specification 

2.2  Operational  Criteria 

2.3  Commercial  Criteria 

2.1  Technical  Specification 

The  prospective  user  will  have  a range  of  known  tasks  which  must  be  performed  in  a 
routine  manner,  some  fringe  tasks  which  he  expects  to  perform  occasionally  and  some 
notion  of  developawnts  which  are  likely  to  be  needed  in  the  future.  Unless  his  tasks 
are  very  straightforward  and  fall  into  a pattern  already  well  established  by  other  users 
it  is  unlikely  that  all  his  present  and  foreseeable  future  requirements  can  be  met 
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adequately  by  one  system.  In  preparing  a specification  the  user  must  consider  his 
priorities  so  that  he  can  grade  features  between  "essential"  and  "desirable  developments" 
and  look  for  development  potential  in  the  system  itself  and  in  the  team  of  people 
supplying  it. 

It  is  not  proposed  to  present  a comprehensive  catalogue  of  features  which  a 
prospective  user  may  seek  in  writing  a specification.  The  following  general  headings 
and  notes  draw  attention  to  some  aspects  of  the  specification  which  may  not  be  immediately 
obvious . 

a ) Types  of  Analysis 

All  users  in  aviation  are  likely  to  requiro  linear  static  analyaia,  eigenvalue 
analysis  and  some  basic  dynamic  capability  and  most  systems  provide  these.  The  other 
features  required  will  depend  on  the  user's  principal  needs  and  priorities.  Basic 
analytical  capability  is  not  easily  added  to  an  existing  system  and  if  for  example  a 
good  non-linear  fracture  mechanics  or  transient  response  capability  is  an  essential 
part  of  the  requirement  the  user  should  not  settle  for  less. 

It  is  very  useful  to  have  a simple  macrolanguage  and  associated  data  structures, 

(e.g.  a matrix  scheme)  such  that  the  user  can  program  his  own  sequences  of  operations  to 
supplement  those  provided  as  standard  and  within  the  system. 

It  may  be  very  convenient  to  be  able  to  perform  non-structural  calculat ions(e .g. 
heat  transfer,  fluid  motion  etc.)  using  the  seme  basic  analysis  formulation  but  very 
often  two  systems,  each  optimised  for  ita  own  function,  will  be  better  then  one. 

b)  Element  Library 


Most  commercial  analysis  systems  have  a good  basic  element  library  and  the  user 
should  look  for  the  simplest  and  most  consistent  family  of  elements  to  perform  the 
functions  he  requires  rather  than  the  largest  and  most  exotic  collection  of  analytical 
frills.  The  ideal  elements  are  usually  those  which  are  simple  to  specify  and  interpret 
and  which  satisfy  reputable  tests  (such  as  the  patch  test)  for  consistency  and  convergence. 

The  day-to-day  user  of  a commercial  analysis  program  is  unlikely  to  be  a finite 
element  analysis  specialist.  Such  users,  in  my  experience,  value  intelligibility  and 
freedom  from  occasional  aberrations  more  highly  than  analytical  refinement.  The  big 
library  is  an  asset,  but  only  if  the  basic  every-day  elements  are  sound. 

c )  Constraints  and  Interconnections 

Most  analysis  systems  have  be.cn  conceived  with  the  objective  of  solving  single, 
well  defined  problems  with  clear-cut  loading  conditions  and  supports  and  a pre-determined 
interconnection  with  other  structures.  Real  structures,  on  the  other  hand,  are  often 
complex  in  their  definition  and  interconnection  and  it  is  frequently  necessary  to  solve 
the  same  basic  problem  with  different  sets  of  boundary  conditions  and  modified  coupling 
between  structures. 

Most  modern  users  will  require  substructuring  facilities  i.e.  the  ability  to  break 
structures  down  into  smaller  units,  to  analyse  them  independently  with  simplified  or 
approximate  boundary  conditions  and  to  interact  adjoining  structures  in  a global  analysis 
when  convenient.  It  is  worth  giving  this  aspect  a good  deal  of  thought  in  specification 
formulation  because  few  systems  are  well  structured  from  this  point  of  view  and  some  are 
extremely  cumbersome  to  use  in  the  substructure  mode. 

It  is  often  helpful  to  take  account  of  symmetries  about  one  or  more  axes  or  planes 
in  formulsting  practical  analyses.  The  user  mav  require  facilities  for  dealing  with 
planar  or  cyclic  symmetry,  (axisymmetry  is  usually  treated  quite  separately  in  a special 
2-dimensional  formulation)  and  for  introducing  symmetric,  antisymmetric  or  repeated 
boundary  conditions.  A particularly  searching  requirement  is  to  link  symmetric  and 
aaystmetric  substructures  by  simple  routines. 

In  many  analyses  it  is  very  convenient  to  be  able  to  Introduce  special  support 
conditions  and  interconnections  or  releases  between  adjacent  degrees  of  freedom. 

Common  requirements  are  to  interconnect  non-coincident  nodes,  to  rigidly  connect  groups 
of  nodes,  to  release  specific  degrees  of  freedom  (mainly  hinges)  and  to  eliminate  near- 
singularities by  coupling  of  freedoms.  User  experience  is  needed  to  appreciate  the 
importance  of  these  points  and  to  frame  a detailed  formal  specification.  It  is  worth 
looking  for  the  following  specific  features: - 

- single-point  constraints  in  any  direction 

- multipoint  constraints  linking  any  number  of  degrees  of  freedom 

- offset  node  and  rigid  elesrant  facilities 

- decoupling  of  specific  degrees  of  freedom  (or  coupling  in  selected  freedoms) 

In  all  the  above  casea  - substructur ing , symmetry  and  constraints -a  good  analysia 
system  will  itself  calculate  all  necessary  connectivity  and  constraint  matrices. 

External  calculation  of  coupling  data,  using  inforauition  already  supplied  in  the  basic 
goesMtry  data,  is  both  tiam-wasting  and  prone  to  serious  errors.  A regular  user  will 
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need  to  add  hia  own  preprocessor  routine  to  a system  which  makee  such  demands. 

d)  Input.  Output  and  Interfaces 

Given  an  adequate  analytical  capability,  the  features  that  will  aioet  affect  the 
ueers  are  the  data  preparation  and  the  output  formats.  Data  preparation  is 
fundamentally  a time-consuming  chore  and  few  people  combine  the  intelligence  and 
experience  needed  for  good  modelling  of  real  problems  with  the  patience  and  care  needed 
to  avoid  errors  when  compiling  routine  data.  Many  analysis  systems  now  have  aids  to 
data  preparation  and  checking  but  these  will  rarely  meet  all  the  needs  of  the  regular 
user  in  a specialised  field.  It  ie  highly  desirable  that  an  analysis  eyetem  should  be 
structured,  both  in  data  formate  and  in  program  architecture,  so  that  the  additional 
pra-  and  post -processor  routines  can  be  added  by  the  user  community.  The  interfaces 
must  be  well-defined  and  stable,  i.e.  they  aiust  not  change  in  any  way  between  one 
version  of  the  main  program  and  another. 

Some  particular  features  which  users  will  find  valuable  are:- 

- Program  and  data  structures  which  permit  breaking  complex  structures  down  into 
convenient,  handleable  unite,  whether  they  are  to  be  analysed  as  substructures 
or  not. 

- Corresponding  presentation  of  intermediate  data  for  checking  and  output  for  ease 
of  interpretation. 

- Unique  epacif icat ion  of  all  physical  data  (l.e.  no  duplication  of  any  physical 
quantities  in  different  blocks  of  data).  In  the  case  of  substructures  this  rule 
mey  be  violated  at  common  nodes  provided  there  is  a clear  hierarchy  of  data 
integrity  or  a fail-safe  checking  procedure. 

- Recovery  of  complete  data  at  the  output  stags  (l.e.  all  the  fundamental  output  such 
as  deflections  or  etreeaee  plus  input  or  intermediate  data  sufficient  to  derive  all 
possible  information  consistent  with  the  analytical  model).  This  feature  is  very 
important  if  the  analyeie  ie  to  be  used  as  part  of  an  automated  design  or 
optimieation  procedure  which  might  require  forme  of  output  not  available  as  standard. 

Input  and  output  consistency  checks;  in  particular  geometry  and  topology  of  the 
input  and  equilibrium  and  compatibility  of  the  output.  (In  dieplacemsnt  enalyeee, 
local  and  global  equilibrium  often  gives  an  excellent  indication  of  numerical 
accuracy) . 

e)  Test  Problems 


An  experienced  user  will  have  discovered  a number  of  tricky  problems  relevant  to 
hie  normal  business  vhich  will  serve  to  check  for  pitfalls  encountered  with  earlier 
systems.  An  alert  new  user  should  also  formulate  some  trial  problems  both  for  geining 
experience  in  data  preparation  and  intrrpretat ion  and  to  find  out  how  competently  an 
analyeie  functions  in  difficult  circumstances.  Some  types  of  problem  which  can  be 
effective  in  showing  up  difficulties  or  inaccuracies  are:- 

- Flexure  of  long,  slender  beams  in  their  plane  - modelled  in  various  ways  using  ber 
and  membrane  elcsiente 

- Flat  plates  under  pressure,  aw>delled  in  various  patterns  using  flexurel  plats 

eleaients 

- Flat  plate  stability,  various  elemint  patterns 

- Large  deflection  of  uniform  slender  beams  with  various  ways  of  modelling 

- Impact  of  rode  and  beams  with  various  modelling  patterns 

The  above  can  all  be  checked  out  amongst  themselves  and  with  standard  solutions  in  the 
literature.  Other  types  of  test  can  be  cerried  out  simply  to  see  whether  eny  solution 
is  obtainable  at  all  and  to  apply  basic  consistency  checks.  The  s»et  searching  taets 
will  be  those  which  either  expose  fundamental  weakneeees  in  current  theory  or  involve 
near-eingularit iee  which  exaggerate  numerical  inaccuracy.  Sosw  examples  are 

- Shallow  shells,  in  particular  shells  analysed  as  membrane  facets  with  one  or  more 
nodes  unsupported  in  the  direction  normal  to  the  surface  (this  can  in  special  cases 
give  true  singularities  which  should  cause  a run  failure)  and  shells  modelled  as 
flexural  elements  where  rotation  about  surfacs  normals  needs  special  treatment. 

Locally  stiff  structures  on  relatively  very  flexible  supports  e.g.  a structure  of 
(stiff)  membrane  eleaunte  simply  supported  on  very  week  springe  (keep  reducing  spring 
stiffness  until  difficulties  or  serious  inaccuracies  arise). 

2.2  Ogerat_ional_Cri,teria^ 

Whilst  the  prospective  user  might  find  it  very  difficult  to  make  a balanced 
Judgement  between  candidate  systems  on  technical  grounds  alone,  the  practicality  of 
using  the  systems  on  specific  computer  installations  can  very  enorsK>ualy.  The  following 
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considerations  are  often  dominant  in  making  the  eventual  decision. 

a)  Computer  Configuration  and  Capacity 

It  can  be  assumed;  at  this  stage,  that  any  system  totally  incompatible  with  the 
users'  computer-configurations  haa  been  eliminated  from  immediate  consideration.  It 
is  important  to  establish  the  operational  limitations,  if  any,  of  candidate  systems  in 
relation  to  all  the  computer  installations  on  which  they  are  to  run.  This  is 
especially  true  whers  different  computers  are  being  used  by  separate  groups  of  users 
in  a single  project.  Without  attempting  an  exhaustive  list,  the  following  ars  soma 
factors  which  must  be  established  for  each  candidate  system 

- Mini  urn  CPU  and  memory  for  efficient  cperation 

- Number  and  capacity  of  supplementary  storage  channels  (disc  packs,  tape  drives  etc) 
for  normal  opsration 

- Other  peripherals  needed  for  normal  execution  and  for  full  exploitation  (e.g. 
plotters,  graphic  stations,  VDU  terminals  stc) 

- Level  of  transportability  of  basic  and  intermediate  data  between  different  computer 
types  (collsborative  situations) 

- Possible  execution  modes  for  (a)  normal  and  (b)  exceptional  slzs  jobs  e.g. 
continuous  batch,  interrupted  batch,  RJE  batch,  RJE  on-line,  interactive  on-line, 
sxternal  buresi  etc. 

Any  deficiencies  of  hardware  configurations  in  relation  to  a given  analysis  systsm 
should  be  expressed  in  simple  terms  such  as  cost-to-remedy.  Such  a cost  dsblt  against 
an  analysis  system  may  well  be  offset  against  improvements  in  technical/operational 
performance  compared  with  other,  less  demanding  systems. 

b)  Execution  Spesd 

Structural  analyses  formulated  in  direct  nodal  geometry,  nodal  loading  and  slement 
data  terms  use  very  large  amounts  of  input  and  output  data  so  that  overall  execution 
times  depend  on  external  data  transmission  as  well  as  mathematical  operations  and 
internal  data  handling.  This  makes  times  and  costs  very  dependent  on  computer  sice 
and  configuration  and  rather  difficult  to  estimate  by  general  algorithms.  It  is  often 
found  that  efficiency  is  also  dependent  on  the  sequencing  of  data  and  this  is  an 
important  factor  to  establish  at  an  early  stage.  The  prospective  user  should  know  not 
only  whether  sensitivity  exists  but  also  whether  the  rules  (or  automatic  rsssquencing 
subroutines)  exist  for  obtaining  efficient  sequences.  If  the  supplier  cannot  answer 
the  question  this  should  be  s warning  that  he  lacks  detailed  knowledge  of  this  system's 
performance  in  practice. 

Again,  structural  analysis,  heat  tr  nsfer  and  particularly  dynamic  response  end  fluid 
stechanics  snalyses  are  often  very  large  individual  calculations.  The  user  should 
specify  the  spproximate  size  of  the  largest  Jobs  ho  can  envisage  and  find  out  whether 
they  can  be  executed  within  the  normal  operating  times  likely  to  be  available.  If 
they  cannot  be  executed  in  a single  pass,  then  efficient  termination  and  restart 
procedurss  are  essential. 

Any  ussr  who  Intends  to  csrry  out  Iterative  calculations  such  as  lsrge  deflection 
snslysls,  transient  snalysls  or  optimisation  must  aim  for  single-pass  execution  time* 
in  seconds  or  minutas  for  normal  size  Jobs  in  order  to  have  any  hope  of  acceptable 
overall  solution  speeds. 

finally,  the  elapeed  time  from  job  conception  to  use  of  results  depends  much  more  on  the 
data  preparat ion, checking  and  Interpretation  times  than  on  the  execution  of  the  analysis 
proper.  The  user  may  wish  to  use  pre-  and  post-processor  routines  supplied  as  part  of 
the  analysis  package  or  add  his  own  routines  in  order  to  spesd  up  and  reduce  errors  and 
tedium  in  the  extraneous  stsges.  In  the  forawr  esse  he  should  give  as  much  attention 
to  evaluation  of  the  routines  available  as  to  the  anslysis  system  Itself.  Their 
simplicity  snd  effectiveness  will  meke  or  mar  the  success  of  the  systsm  as  s whole. 

If  the  user  wishes  to  sdd  his  own  routines  (and  most  will  eventually  reach  this  position) 
then  it  cannot  be  over-emphasised  that  the  structure  of  the  bssic  program  and  the  inter- 
faces must  be  euch  as  to  give  the  user  ready  access  to  luterstedlate  dsts  and,  as  already 
sient ioned,  they  must  remain  stable  as  the  program  evolves  to  svoid  obsolescence  or,  worse 
still,  lnsccuracy  in  the  future. 

c ) Evolution  snd  Support 

The  state  of  the  art  in  finite  elesient  analysis  is  still  chsnglng  rapidly  and  a 
system  which  looks  good  todav  stay  seem  awdlocre  tostorrow.  Llkewlee,  with  computer  hard- 
ware, the  system  which  functions  well  on  today's  coaiputers  mey  be  quits  inappropriate  to 
the  next  generation.  Reprogramming  siejor  systema  for  incompatible  hardware  has  been 
psrhaps  the  biggest  headache  for  computer  users  in  the  past  two  decades.  Whilst  many 
advances  have  been  made  in  Interchangeable  software,  the  problem  is  still  with  us  in 
relation  to  the  'architecture*  of  central  prograau  and  data  bass  stenagesient  to  make  best 
use  of  the  present-day  hardware.  We  are  now  approaching  one  of  the  aujor  watersheds  in 
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technical  computing  - the  transition  from  mainframe-dominated  multi-user  aystema  to 
distributed  computing  using  linked,  more  apecialised  machines  with  their  own  data  baaea. 

The  largest  and  most  active  finite  element  software  auppliera  are  recogniaing  this 
and  are  adapting  their  aystema  accordingly.  This  is  but  one  example  of  major  evolution 
which  ia  prohibitively  expensive  for  individual,  in-house,  development  but  which  can  be 
tackled  economically  by  the  multi-cuatomer  aupplier.  In  thia  reapect  one  must  judge 
suppliera  by  their  paat  performance  and  by  their  responsiveness  to  proposed  change. 

When  all  the  other  criteria  have  been  established  and  several  candidate  aystema  atill 
remain  thia  may  be  the  beat  area  for  personal  judgement.  The  prospective  users  and 
their  managers  should  meet  the  supplier  team,  diacusa  their  present  and  future  plans, 
evaluate  the  health  and  vigour  of  their  user  support  services  and  make  what  can  only  be 
a subjective  Judgement  of  their  likely  ability  to  move  with  the  times  without 
disrupting  continuity. 

Likewise,  when  things  go  wrong,  aa  ia  inevitable  in  any  dynamic  system,  the 
supplier  team  should  be  competent  and  able  to  come  to  the  aid  of  the  user.  The  cost 
and  management  of  the  total  computer  analysis  facility  will  be  quite  different  if  the 
user  can  rely  on  competent  proieasional  support  when  in  difficulties  rather  than  have 
to  build  up  hia  own  support  team  to  cope  with  all  possible  ariaings.  This  ia  the  main 
reason  why  commercial  software  suppliers  are  usually  much  more  effective  than  informal 
software  sharing  schemes.  If  the  supplier  takes  responsibility  for  the  integrity  of 
his  product  this  relieves  the  user  of  a large  insurance  investment  or  of  the  delays 
involved  in  "fire-brigade"  actions  to  rectify  unexpected  errora.  Points  to  look  for 
here  are  the  existence  of  an  active  user  community  and  a well  organised  system  for 
disseminating  information  on  user  problems  and  their  solutions.  However,  too  frequent 
updates  of  basic  programs  for  error-correction  (aa  opposed  to  genuine  evolution)  should 
be  a warning  as  to  the  competence  of  the  supplier's  team. 

One  further  important  aspect  of  support  concerns  the  documentation.  Any  purchaser 
or  hirer  of  a computing  system  should  be  able  to  stand  on  his  own  as  regards  normal  use 
of  the  system  after  initial  familiarisation.  Documentation  must  be  comprehensive  and 
embrace  the  engineering  user,  the  specialist  programmer  and  the  system  support  team 
(where  required).  It  should,  hopefully,  be  easy  to  read  and  be  well  indexed  and  croas- 
refsrencsd  for  easy  accsss  to  information.  These  last  points  are  deficiencies  in  moat 
known  documentation,  the  main  difficulty  being  that  manuals  are  written  by  involved 
special ists  and  read  by  non-apacialista  - a classic  recipe  for  a communication  gap.  It 
would  be  an  outstanding  recommendation  of  the  competence  of  any  team  if  it  could  supply 
documentation  which  waa  at  the  same  time  technically  rigorous  and  wholly  intslligibls 
to  non-spscialist  engineers. 

d)  Prior  Experience 

If  one  or  more  partners  in  a joint  project  already  have  experience  of  the  uae  of  a 
particular  system  this  is  bound  to  figure  prominently  in  any  asssssmsnt.  It  may  have 
either  a positive  or  negative  influence  according  to  how  satisfied  the  users  fsel  with 
the  system  they  know.  Unnecessary  disruption  of  an  edsquate  and  efficient  system  In 
operation  is  a managerial  crims.  Equally,  it  is  folly  not  to  recognise  shortcomings 
in  a known  system  and  uss  these  as  benchmarks  for  judging  others.  Either  way, 
experience  is  a valuable  asset  and  should  bs  heavily  weighted  in  final  assessment. 

2.3  Commercial  Criteria 


Under  this  heading,  direct  coat  of  buying  or  leasing  programs  is  likely  to  bs  a 
relatively  minor  consideration.  Ths  market  is  so  competitive  that  any  comswrcial  system 
looks  cheap  compared  with  the  in-houss  investment  which  would  be  needed  to  emulate  it. 
Much  mors  important  srs  ths  support  and  running  coats  of  ths  system,  any  constraints  on 
its  use  and  guarantees  of  future  availability,  and  support. 

a)  Costs 

Factors  to  be  considered  in  comparing  different  systems  may  include  ths  following 

- First  cost  or  initial  entry  cost  of  the  basic  system 

- Periodic  rental  of  ths  basic  system 

- Supplementary  software  costs 

- Service  and  maintenance  costs 

- ^Contributions  to  new  development^ 

- In-houss  support  costs 

- Running  costa 

- Costs  of  computer  enhancement  to  embody  the  eystsm. 

Contributions  to  new  development  are  only  valid  for  Inclusion  at  this  stage  if  they 
are  necessary  to  bring  a system  up  to  a competitive  standard.  In-houss  support  and 
running  costs  are  the  aw>at  difficult  itsma  to  estimate  in  advance  sid  will  therefore 


repay  the  nest  careful  attention;  here  la  another  case  where  the  experience  of  other 
users  is  invaluable. 

b)  Legal  and  Commercial  Linitationa 

Buying  or  hiring  a commercial  analysis  system  involves  a large  measure  of  dependency 
of  the  user  on  the  supplier  and  this  in  turn  requires  a good  commercial  relationship 
between  the  two  parties.  Whilst  the  supplier  can  impose  few  legally  enforcible 
restrictions  on  the  user  he  may  well  make  quite  stringent  contractual  limitations  which 
it  is  in  the  user's  interest  to  observe.  For  example,  the  use  of  the  system  is  likely 
to  be  limited  to  a single  site  or  even  a single  computer  installation  unless  a special 
deal  involving  several  sites  is  negotisted. 

The  supplier  may  limit  the  user's  access  to  the  basic  source  programs  to  prevent 
tampering  with  the  internal  workings  of  the  system  which  might  in  turn  invalidate  any 
guarantees  of  integrity.  On  this  topic,  it  is  as  well  to  establish  from  the  outset, 
what  liability  the  supplier  will  accept  for  deficiencies  in  the  programs  supplied.  A 
good  supplier  will  usually  undertake  to  make  good  any  fundamental  system  defect 
discovered  by  a user  at  his  (the  supplier ' s lexpense . He  is  unlikely  to  reimburse  any 
costa  Incurred  by  the  user  in  failing  to  achieve  a correct  result. 

From  the  user's  viewpoint,  the  most  important  criterion  under  this  general  heading 
is  likely  to  be  continuity.  A user  will  become  involved  with  an  analysis  system  as  a 
way  of  life  and  it  becomes  increasingly  difficult  to  change  rapidly  from  one  system  to 
another.  Any  user  is  likely  to  need  a guarantee  of  on-going  support  for  a system  at 
least  12  months  ahead;  assurances  of  development  and  support  over  far  longer  periods, 
say  5 years,  are  needed  as  a basis  for  proper  planning.  These  factors  are  especially 
important  when  obtaining  a bureau  service  which,  in  theory,  could  terminate  overnight. 

In  the  collaborative  project  field  a user  requires  an  on-going  commitment  on  the 
part  of  his  partners.  Assurances  are  likely  to  be  easily  obtained,  guarantees  are 
unlikely.  In  any  event  a consensus  agreement  on  a system  is  required  rather  than  a 
unilateral  one.  This  brings  us  back  to  a very  difficult  issue  and  one  which  can  have 
a serious  impact  on  real-life  decisions.  Consensus  between  partners  in  different 
companies  and  different  countries  cannot  be  divorced  from  feelings  of  national  or 
corporate  loyalty  or  even  political  pressure.  If  such  considerations  are  likely  to 
prove  important  they  should  be  identified  before  committing  too  much  effort  in  a 
pseudo-scientific  evaluation. 

3.  MANAGEMENT  DECISION 


The  person  or  team  responsible  for  assessing  competing  systems  will  have  compared 
candidate  systems  on  the  basis  of  some  or  all  of  the  above-mentioned  criteria,  together 
with  others  which  may  relate  to  their  special  circumstances.  Unless  a clear  choice 
emerges  to  the  satisfaction  of  all  concerned,  some  formal  comparison  may  be  required. 

It  is  suggested  that  this  should  take  one  of  two  forms  - a weighted  quantitative 
comparison  of  system  features,  or  an  effective  cost  summary.  In  some  cases  the  two  may 
be  combined,  as  previously  indicated,  into  a single  figure  of  merit  for  each  system 
together  with  some  measure  of  sensitivity  to  the  more  important  factors.  This  is  the 
information  on  which  Judgment  should  be  based  and  if  the  formal  evaluation  still  remains 
finely  balanced  it  cannot  matter  very  greatly  which  system  is  chosen.  A thoroughly 
subjective  Judgment  based  on  personal  preference  or  confidence  is  quite  in  order. 

The  paper  concludes  with  a hypothetical  example  to  illustrate  some  of  the  points 
mode  above.  The  formal  assessment  chart  in  Fig. 2 lists  only  the  main  headings  used  in 
the  previous  text  end  each  feature  is  first  ranked  on  a scale  of  0-3  according  to  the 
extent  to  which  on  analysis  system  satisfies  the  more  detailed  criteria  under  each 
heeding.  A weighting  factor  is  used  egainst  each  feature  which  is  multiplied  by  the 
ranking  number  to  give  a rating  which  is  then  accumulated  over  all  the  features. 

In  this  hypothetical  case  a simple  and  cheep  system  A is  compared  with  two  more 
sophisticated  systems  - B being  particularly  strong  in  its  operational  performance  and 
C in  technical  performance. 

The  features  rated  in  the  chart  are  divided  into  three  groups  to  assist  in 
subsequent  judgment.  The  first  consists  of  factors  which  ere  mosly  factual  and  where 
the  ranking  sequence  can  be  established  with  confidence.  It  is  mainly  the  weighting 
factor  which  is  open  to  argument  end  the  sensitivity  to  this  can  be  assessed  by  trying 
different  combinations  of  reesonsble  factors.  In  the  case  shown  it  makes  no 
significant  difference  whether  the  two  weighting  levels  are  used  or  ell  the  rankings  ere 
used  without  weighting.  This  particular  facet  of  judgment  is  therefore  not  unduly 
sensitive. 

The  second  group  consists  of  features  whose  ranking  depends  on  e subjective  asaesa- 
OMnt  either  of  future  happenings  (evolution,  legal  constraints  etc)  or  of  the  value  of 
an  ad-hoc  comparison  based  on  necessarily  incomplete  or  inconsistent  evidence  (test 
problems  and  user  experiences).  The  weighting  of  these  numbers  involves  e further 
Judgawnt  on  the  pert  of  the  compiler  and  hence  the  sensitivity  of  these  factors  to 
personal  bias  is  clearly  higher  then  for  the  previous  group. 

Finally,  cost  has  been  kept  separate  so  as  to  isolate  the  other  type  of  value  judgaMnt 
namely  the  comparative  value  of  all  the  other  features  with  basic  costs  as  defined  in 
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the  text.  This  value  is  amenable  to  analysis  if  it  is  possible  to  put  a cost  figure 
against  the  enhancement  of  technical  and  operational  performance  to  meet  the  required 
standards  or  the  direct  cost  implication  of  sub-standard  performance  (e.g.  execution 
speed).  In  this  case  the  alternative  presentation  of  Pig. 3 is  likely  to  be  more 
meaningful . 

From  Fig. 2,  clearly  system  A would  be  rejected  unless  the  importance  of  cost  has 
been  seriously  underestimated.  Equally  clearly  the  difference  shown  between  systems  B 
and  C is  not  significant  in  relation  to  the  possible  inaccuracies  of  many  parts  of  the 
assessment.  This  same  conclusion  is  reached  if  we  make  very  different  estimates  of 
weighting  factors.  Inspection  of  the  chart  narrows  down  the  field  of  judgment  for  the 
mansger  to  the  following  principal  issues 

Superior  range  of  analysis  capabilities  and  computer  compatibility  for  system  B 

vs  superior  input/output  and  ir‘«rface  stability  for  C. 

Better  user  experience  and  com  .ence  in  the  development  team  for  system  B. 

Lower  cost  for  system  C. 

Assuming  that  the  relative  weighting  of  the  technical  issues  is  reasonably  accurate 
the  final  judgment  comes  down  to  the  value  placed  on  confidence  in  the  supplier  and  his 
team. 


The  cost  summary,  as  illustrated  in  Fig. 3 is  superficially  far  more  satisfactory 
than  the  rather  arbitrary  points  assessment.  However  it  is  usually  not  possible  to 

make  the  type  of  cost  forecasts  needed  without  extensive  experience  of  the  system  before- 
hand. Furthermore,  the  difference  in  cost  to  the  user  of  modifying  or  extending  a 
complex  system  himself  as  opposed  to  doing  the  same  job  with  the  assistance  of  the 
supplier  could  be  very  large.  But  the  major  deficiency  of  a cost  comparison  alone  is 
that  it  does  not  reflect  confidence  in  the  integrity  of  the  system,  the  supplier's 
ability  to  maintain  and  develop  it  or  the  continued  availability  of  the  system  except 
in  so  far  as  these  are  reflected  in  the  allot/ances  made  for  enhancement  and  support. 

The  figures  shown,  whilst  wholly  fictitious  do  illustrate  some  important  points,  of 
which  the  most  obvious  is  the  relatively  small  proportion  of  the  total  operating  cost 
which  is  attributed  to  first  cost  or  rental.  The  level  of  S-10K  shown  here  is  probably 
quite  typical  of  a user  with  a large  workload  - and  these  costs  only  cover  the  analysis 
system  itself  and  its  direct  computer/programming  support, not  ihe  engineering  costs  of 
job  execution.  The  figures  have  been  made  broadly  consistent  with  the  ratings  in  Fig. 2 
so  that  a cost  advantage  is  shown  for  system  C (it  requires  less  technical  enhancement 
and  has  lower  running  costs).  This  is  to  be  offset  against  the  subjective  factors  of 
Fig. 2 which  clesrly  favoured  system  B and  which  imply  that  supplier  assistance  could  be 
wxpected  to  cut  back  the  development  costs.  So  the  same  fundamental  management 
decision  remains;  the  terms  in  which  it  is  presented  (money  versus  confidence)  sre 
rather  more  clear  cut  but  no  more  accurate. 

Returning  to  a point  made  at  the  beginning,  it  is  likely  to  be  more  cost  effective 
to  make  a judgment  at  this  stage  rather  than  mount  a detailed  comparative  study  (by 
perhaps  giving  both  systems  a limited  trial  run)  and  defer  decision.  The  time  spent  in 
appraisal  would  be  better  used  in  enhancing  deficiencies  in  the  chosen  system. 


19 


ANALYSIS  SELECTION  PROCESS  0£J 


CANDIDATE 

SYSTEMS 


Tnitial 

CRITERIA 

.Go  /NoG 


TECHNICAL 

SPECIFICAHOK 

OPT  RATIONAL 
CRITERIA 

COMMERCIAL 

CRITERIA 

1 

zn 

r~ 

1 

REJECT 


l sptoble 


CHOSEN  SYSTEM 


20 


FORMAL  ASSESSMENT  CHART 


YPES  OF  ANALYSIS 


LEMENT  LIBRARY 


ONSTRAINTS.  CONNECTIONS 


0 AND  INTERFACES 


OMPUTER  COMPATIBILITY 


XECUTION  SPEED 


OBJECTIVE  FACTORS 


EST  PROBLEMS 


VOLUTION.  SUPPORT 


SERS'  EXPERIENCE 


LEGAL/COMMERCIAL 

CONSTRAINTS 


SUBJECTIVE  ASSESSMENTS 


SYSTEM  RATING 


COST  RATING 


COST  EFFECTIVENESS 
RATING 


RANK  3 
KEY  j 
0 


EXCELLENT 

GOOD 

FAIR 

POOR 


WEIGHT 

FACTOR 


ALL  CRITERIA  SATISFIED 
IMPORTANT  CRITERIA  SATISFIED 
ENHANCEMENT  DESIRABLE 
EXTENSIVE  ENHANCEMENT  ESSENTIAL 


FIG,  3 

COST  COMPARISON 

CHART 

BASIC  COST 

ELEMENT 

SYSTEM  B 

SYSTEM  C 

FIRST  COST.  INITIAL  ENTRY  COST 

tlOK 

£50K 

BASIC  RENTAL 

SUPPLEMENTARY  SOFTWARE 

£30K 

0 

C30K 

0 

SERVICE  AND  MAINTENANCE 

BASIC  RUNNING  COST 

£20K 

C20K 

£(l50K 

1300K 

IN-HOUSE  SUPPORT 

£200K 

£1  20K 

TOTAL  BASIC  COSTS 

£f<inK 

£4£OK 

ENHANCEMENT 

COSTS 

SYSTEM  B 

SYSTEM  C 

EXTEND  ANALYSIS  TYPES  • 
EXTEND  ELEMENT  LIBRARY  • 

XDD  CONSTRAINT  FACILITIES  * 
ADD  A MAINTAIN 

PRE-  AND  POST-PROCESSORS 
JPGRADE  COMPUTER  HARDWARE 


£ISOK 

E6o 

£70 


TOTAL  ENHANCEMENT  COSTS 


COST  TO  SATISFY  SPECIFIED  CRITERIA 


COSTS  IF  WHOLE  JOB  IS  UNDERTAKEN  BY  THE  USER  ALONE 
(REDUCED  IF  SUPPLIER  AND/OR  PARTNERS  COOPERATE). 


£<|R0K 


£l 22CK 


£5f»OK 


£1070K 


REPORT  DOCUMENTATION  PAGE 


(.Recipient’s  Reference 

2.  Originator's  Reference 

3.  Further  Reference 

4.  Security  Classification 
of  Document 

AGARD-R-670 

ISBN  92-835-1305-3 

UNCLASSIFIED 

S. Originator  Advisory  Group  for  Aerospace  Research  and  Development 


North  Atlantic  Treaty  Organization 
7 rue  Ancelle,  92200  Neuilly  sur  Seine,  France 

6^  Title 

SELECTION  OF  STRUCTURAL  ANALYSIS  ^ 
COMPUTER  PROGRAMS 


7.  Presented  at 


8.  Aut  horf  s)/Edit  orf  s) 


47th  Structures  and  Materials  Panel  Meeting, 
Florenee,  Italy,  September  1978. 


L.V. Andrew  and  I.C.Taig 
1 10.  Author's/Editor's  Address 


See  Flyleaf 


T 9.  Date 

January  1979 

11.  Pages 

26 


12.  Distribution  Statement  T|,js  jocumcnt  is  distributed  in  accordance  with  AGARD 

policies  and  regulations,  which  are  outlined  on  the 
Outside  Back  Covers  of  all  AGARD  publications. 

1 3 . Key  words/ Descrip  tors 


Computer  programs 
Structural  analysis 


< 15.  Abstract 

J 

The  use  ot  computerised  techniques  of  structural  analysis  is  now  standard  in  many  branches  of 
engineering.  There  is,  however,  a wide  range  of  programs  available  both  commercially  and 
within  individual  organisations.  These  programs  differ  in  their  capabilities  and  in  their  costs 
and  ease  of  use.  The  potential  user  may  experience  considerable  difficulty  in  selecting  a 
program  that  is  appropriate  to  his  particular  class  of  work. 

The  paper  by  Mr.  Anurew  describes  in  detail  the  technical  and  administrative  course  of  action 
that  has  been  adopted  by  a major  industrial  organisation  to  select  and  implement  programs  that 
are  appropriate  to  its  work.  Mr.  Taig  presents  a similar  discussion  but  with  perhaps  more 
emphasis  on  technical  issues. 

| 

Papers  presented  at  the  47th  Structures  and  Materials  Panel  Meeting,  Florence,  Italy, 

September  1978. 


Computer  aided  design 
Computers 


as . 

is1. 


Is 

i 


NATO  ^ OTAN 

7 RUE  ANCELLE  • 92200  NEUILLY-SUR-SEINE 
FRANCE 


DISTRIBUTION  OF  UNCLASSIFIED 
AGARD  PUBLICATIONS 


Telephone  745.08.10  • Telex  610176 


ACARD  does  NOT  hold  stocks  of  AGARD  publications  at  the  above  address  for  general  distribution.  Initial  distribution  of  AGARD 
publications  is  made  to  AGARD  Member  Nations  through  the  following  National  Distribution  Centres.  Further  copies  are  sometimes 
available  from  these  Centres,  but  if  not  may  be  purchased  in  Microfiche  or  Photocopy  form  from  the  Purchase  Agencies  listed  below. 


BELGIUM 

Coordonnateur  AGARD  - VSL 
Etat-Major  de  la  Force  Afrienne 
Quartier  Reine  Elisabeth 
Rue  d’Evere,  1 140  Bruxelles 


NATIONAL  DISTRIBUTION  CENTRES 
ITALY 

Aeronaut  ica  Militare 

Ufficio  del  Delegato  Nazionale  all’ AGARD 
3,  Piazzale  Adenauer 
Roma/ EUR 


CANADA 

Defence  Scientific  Information  Service 
Department  of  National  Defence 
Ottawa,  Ontario  K1 A OZ2 

DENMARK 

Danish  Defence  Research  Board 
Qsterbrogades  Kaserne 
Copenhagen  (J 

FRANCE 

O.N.E.R.A.  (Direction) 

29  Avenue  de  la  Division  Leclert 
92  Chitillen  sous  Bagneux 


LUXEMBOURG 
See  Belgium 

NETHERLANDS 

Netherlands  Delegation  to  AGARD 
National  Aerospace  Laboratory,  NLR 
P.O.Box  126 
Delft 

NORWAY 

Norwegian  Defence  Research  Establishment 
Main  Library 
P.O.  Box  25 
N-2007  Kjelier 


GERMANY 

Zentralstelle  fur  Luft-  und  Raumfahrt- 
dokumentation  und  -information 
c/o  Fachinformationszer.lrum  Energie, 
PhysiV,  Mathematik  GmbH 
Kernforschungszentrum 
7514  Fggenstein-Leopoldshafen  2 

GREECE 

Hellenic  Air  Force  General  Staff 
Research  and  Development  Directorate 
Holargos,  Athens,  Greece 


ICELAND 

Director  of  Aviation 
c/o  Flugrad 
Reykjavik 


UNITED  STATES 


PORTUGAL 

DireceSo  do  Servico  de  Material 

da  Forca  Aerra 

Rua  da  Escola  Politecnica  42 

Lisboa 

Attn:  AGARD  National  Delegate 
TURKEY 

Department  of  Research  and  Development  (ARGE) 
Ministry  of  National  Defence,  Ankara 

UNITED  KINGDOM 

Defence  Research  Information  Ctntre 
Station  Square  House 
St.  Mary  Cray 
Orpington.  Kent  BR5  3RF. 


National  Aeronautics  and  Space  Administration  (NASA) 
^ Langley  Field,  Virginia  23365 
**■  Attn:  Report  Distribution  and  Storage  Unit 


TUI  UNITED  STATES  NATIONAL  DISTRIBUTION  CENTRE  (NASA)  DOFS  NOT  HOLD 
STOCKS  OF  AGARD  PUBLIC  ATIONS.  AND  APPI  1C  ATIONS  FOR  COPIES  SHOULD  BE  MADE 
DIRECT  TO  THE  NATIONAL  TECHNICAL  INFORMATION  SERVICE  (NTIS)  AT  THE  ADDRESS  BELOW. 


Microfiche  or  Photocopy 
National  Technical 
Information  Service  (NTIS) 
5285  Port  Royal  Road 
Springfield 
Virginia  22161.  USA 


PURCHASE  AGENCIES 
Microfiche 

Space  Documentation  Serv  ice 
European  Space  Agency 
10,  rue  Mario  Nikis 
75015  Paris.  France 


Microfiche 
Technology  Reports 
Centre  (DTI) 

Station  Square  House 
St  Mary  Cray 
Orpington,  Kent  BR5  3RF 
England 


Requests  for  microfiche  or  photocopies  of  AGARD  S xuments  should  include  the  AGARD  serial  number,  title,  author  or  editor,  and 
publication  date.  Requests  to  NTIS  should  include  the  NASA  accession  report  number,  Full  bibliographical  references  and  abstracts 

of  AGARD  publianons  are  given  in  the  following  journals. 


Scientific  and  Technical  Aerospace  Reports  (STAR) 
published  bv  NASA  Scientific  and  Technical 
imormaiion  f acility 
Post  Office  Be  x STS7 

Baltimore  Washington  International  Airport 
Maryland  2 1 240,  USA 


Government  Reports  Announcements  (GRA) 
published  by  the  National  Technical 
Information  Services,  Springfield 
Virginia  22161,  USA 


Printed  by  Tcchricti  Editing  end  Reproduction  Ltd 
Harford  Home , .’-  S Charlotte  St,  London  WtP  !HD 


